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Benefits  of  Using  the  Photosimulation  Laboratory  Environment  for  Camouflage 

Assessment 


Thomas  J.  Meitzler,  David  Bednarz,  Euijung  Sohn,  Darryl  Bryk,  and  Kimberly  Lane 

Survivability  Technology  Area 
U.S.  Army  Tank-automotive  and  Armaments  Command 
Warren,  MI,  48397-5000 


ABSTRACT 

A  method  is  described  for  using  the  photosimulation  laboratory  environment  to  evaluate  the  effectiveness  of 
camouflage  for  military  vehicles.  There  are  distinct  advantages  to  acquiring  images  at  the  field  site  and  then  bringing  them 
back  for  observer  testing  in  a  laboratory  environment.  Laboratory  testing  provides  a  repeatable,  secure,  and  low-cost  way  to 
generate  realistic  performance  data  for  vehicle  evaluation  for  the  purposes  of  signature  testing,  measurement  of  the 
effectiveness  of  camouflage  relative  to  a  baseline  vehicle,  and  calibration  and  validation  of  target  acquisition  models.  A  test 
is  described  by  the  authors  in  which  a  baseline  LAV  is  compared  to  a  treated  LAV  in  the  TARDEC  Visual  Perception 
Laboratory  using  imagery  collected  from  the  field  in  the  manner  prescribed  by  an  experimental  design. 


INTRODUCTION 

High-resolution  digital  cameras  presently  available  on  the  market  are  rapidly  approaching  the  quality  of  film 
resolution.  Five  and  six  megabyte  CCD  imaging  chips  in  combination  with  the  ability  to  capture  imagery  in  raw  24-bit 
format,  combined  with  large  memory  storage  devices  enable  high  resolution  imagery  to  be  captured  at  field  site  locations. 
Using  high-resolution  graphics  projectors,  the  imagery  can  be  presented  in  the  controlled  environment  of  the  lab  in  such  a 
manner  as  to  obtain  observer  data  with  confidence  levels  approaching  99%.  It  is  the  authors  opinion  that,  the  benefits  of  the 
high  degree  of  confidence  achieved  using  the  repeatability  offered  by  the  lab  environment,  far  outweighs  any  perceived 
advantage  of  having  multiple  teams  of  observers  present  at  the  field  location  to  search  for  vehicles. 

The  Light  Armored  Vehicle  (LAV)  Family  of  Vehicles  (FOV)  was  developed  to  provide  the  Marine  Corps  with 
enhanced  mobile  warfare  capabilities.  General  Motors,  Diesel  Division  in  London,  Ontario,  Canada,  began  manufacturing 
the  LAV  FOV  in  1982  and  completed  delivery  to  the  USMC  in  April  1988.  The  LAV  FOV  includes  several  variants  which 
utilize  light  armor  protection  from  small  arms,  light  machine  gun  fire,  artillery  projectile  fragments,  and  mine  fragments. 
Each  variant  was  designed  for  a  specific  mission  function  and  was  mounted  on  a  common  chassis.  The  LAV  FOV  consists 
of  direct  fire  variants  armed  with  25mm  guns  (LAV-25),  with  TOW  II  Missiles  on  LAV  Anti-Tank  (LAV- AT),  an  Air 
Defense  variant  (LAV- AD)  capable  of  providing  air  defense  coverage  for  the  LAR  unit,  and  support  variants  consisting  of 
Command  and  Control  (LAV-C2),  Mortar  (LAV-M),  Logistics  (LAV-L),  and  Recovery  (LAV-R)  vehicles. 

The  LAV  is  a  highly  mobile  vehicle  for  conducting  reconnaissance,  counter  reconnaissance,  limited  offensive  and 
defensive  operations  and  other  missions.  The  current  LAV  is  increasingly  difficult  to  maintain  at  acceptable  levels  of 
readiness  as  it  nears  the  end  of  its  original  projected  service  life.  Budget  constraints  prevent  the  development  and  fielding  of 
a  new  vehicle  capable  of  performing  its  assigned  missions.  The  main  purpose  of  the  Light  Armored  Vehicle  Service  Life 
Extension  Program  (LAV-SLEP)  is  to  address  the  major  subsystems  that  are  currently  degrading  readiness  or  are  driving 
higher  than  necessary  O&S  costs.  The  LAV  retains  the  same  requirements  for  performance  capabilities  and  characteristics  in 
the  areas  of  mobility,  lethality,  survivability,  and  sustainability. 

The  test  described  in  this  report  was  undertaken  as  part  of  the  LAV-SLEP.  Specifically,  the  purpose  of  this  field 
test  was  to  determine  the  performance  of  a  treatment  known  as  ADCAM  in  reducing  the  probability  of  detection  in  the  visual 
part  of  the  electromagnetic  spectrum  at  unity  magnification  (as  seen  by  the  naked  eye)  at  various  ranges,  aspect  angles  and 
lighting  conditions.  Only  the  baseline  results  will  be  described  in  this  report  because  comparisons  of  the  treated  versus 
untreated  are  classified. 


METHOD 

The  test  design  implemented  was  an  extension  of  visual  detection  requirements  provided  to  our  lab  from  PM  LAV. 
Initially  the  PM  requested  only  two  range  points  at  standard  engagement  ranges.  We  suggested  having  more  ranges  in 
between  the  critical  range  points  to  obtain  a  probability  of  detection  versus  range  curve,  as  is  more  typical  for  these  kinds  of 
tests.  A  test  matrix  was  developed  in  full-factorial  form  and  24-bit  color  imagery  was  collected  using  a  Kodak  460  digital 
camera.  The  images  were  prepared  for  the  photosimulation  test  and  then  presented  to  30  subjects.  The  experimental  factors 
and  levels  with  their  values  are  shown  below  in  Table  1 .  The  photosimulation  test  in  the  lab  was  arranged  so  that  a  pixel 
IFOV  subtended  by  the  display  was  less  than  one  minute  of  arc  and  the  displayed  image  represented  a  unity  magnification  or 
IX  representation  to  the  subject.  The  first  test  was  meant  to  emulate  naked-eye  vision.  Prior  to  the  actual  test,  the  subjects 
were  instructed  on  the  purpose  of  the  test  as  well  as  required  to  take  a  pre-test  in  which  they  could  become  familiar  with  the 
imagery  and  software.  None  of  the  pictures  used  in  the  pre-test  were  used  in  the  actual  test,  however,  the  images  were  from 
the  same  set.  The  test  procedure  was  to  display  an  image  with  a  time-out  of  30  seconds.  The  imagery  is  cropped  so  that  no 
scrolling  is  required.  The  target  can  appear  within  one  of  five  possible  regions.  The  soldier  must  use  the  mouse  to  “click-on” 
what  her  or  she  thinks  is  a  target,  based  on  the  training. 

Analysis  of  the  first  test  showed  most  subjects  obtained  a  score  of  only  20  %  detection.  This  is  not  unreasonable 
given  the  difficulty  of  the  imagery.  The  ranges  are  not  unusual  for  such  a  test,  however  the  high  degree  of  clutter  and  in 
particular  the  height  of  the  grass  on  the  terrain  makes  it  difficult  for  the  unaided  eye  to  detect  common  cure  features  of  the 
vehicle.  A  second  test  was  arranged  at  a  power  of  3X.  The  imagery  from  the  field  was  of  sufficient  resolution  so  that  there 
was  no  noticeable  increase  in  pixelation  of  the  imagery  an  in  increase  in  magnification.  The  presentation  in  the  lab  was 
randomized,  this  is  a  very  good  reason  to  use  the  lab. 


Region 

1 

Top-Left 

2 

Top-Right 

3 

Lower-Left 

4 

Lower-Right 

5 

Center 

Vehicle  Type 

1 

Baseline  (old  LAV) 

2 

SLEP  +  ADCAM 

3 

SLEP  +  ADCAM  - 

ADCAM  bowplane 

Aspect  angle 

1 

Front 

2 

30  degree 

3 

Side 

Lighting 

1 

Front  Lit 

2 

Back  Lit 

Weather  condition 

1 

Clear 

2 

Overcast 

Range  (km) 

i 

1 

2 

1.5 

3 

2 

4 

2.5 

5 

3 

Table  1 :  Factor  matrices  for  the  visual  detection  test 


At  this  point,  it’s  instructive  to  digress  for  a  moment  to  emphasize  the  rationale  for  using  the  type  of  design  methodology 
described  by  the  authors  in  this  paper.  Statistically  based  experimental  design  is  a  strategy  of  designing  experiments  in  such  a 
manner  as  to  develop  a  robust  test  plan.  In  other  words,  a  test  plan  that  is  minimally  affected  by  external  sources  of 
variability.  What  makes  vehicle  field  test  challenging  is  that  there  are  many  variables  that  are  present  and  must  be  accounted 
for.  In  addition,  the  variables  interact  with  each  other.  The  correct  approach  to  working  with  several  factors  is  to  conduct  a 
factorial  experiment.  A  factorial  experiment  is  an  experimental  strategy  in  which  the  factors  are  varied  together,  rather  than 
one  at  a  time.  The  factorial  experimental  design  concept  is  extremely  important  and  powerful  when  used  correctly.  Entire 
books  are  written  on  how  to  choose  an  experimental  design  based  on  considerations  such  as  time,  money  and  availability  of 
materials.  Montgomery  wrote  a  book  that  is  a  standard  in  the  field  of  experimental  design.1  For  the  type  of  tests  that  the 
authors  are  involved  with,  it  is  dangerous  to  leave  out  data  points  in  the  hope  of  making  the  test  plan  more  expedient. 
Attempts  to  do  just  that  in  the  past  have  resulted  in  things  such  as  the  ‘heartbeat  effect’  in  data.  The  range  of  variability  in  the 
factors  is  so  great  and  interdependent,  that  experimental  designs  such  as  fractional  factorial  designs  are  to  be  avoided. 

When  making  inferences  about  differences  in  a  factor  in  a  perception  experiment  in  the  laboratory  we  want  to  make  the 
experimental  error  as  small  possible.  This  requires  that  we  remove  the  variability  between  subjects  from  the  experimental 
error.  The  design  we  use  to  accomplish  this  is  a  factorial  experiment  run  in  a  randomized  complete  block.  By  using  this 
design  with  the  subjects  as  blocks  we  form  a  more  homogeneous  experimental  unit  on  which  to  compare  different  factors. 
This  experimental  design  improves  the  accuracy  of  the  comparisons  among  the  different  factors  by  eliminating  the  variability 
among  the  subjects.  Within  a  block,  the  order  in  which  the  treatment  combinations  are  run  is  randomly  determined.  It  is 
usually  not  possible  to  implement  this  experimental  design  in  the  context  of  a  traditional  field  test. 


The  pictures  below  in  Fig.  1  through  Fig.  6  were  used  for  training  observers  as  to  what  kind  of  vehicles  they  would  be 
looking  for. 


Fig.  1:  Baseline  side  Fig.  2:  ADCAM  side 


The  charts  in  Fig.  8  below  show  the  results  of  measuring  the  X  and  Y  chromaticity  values  of  the  monitors  that  were  used  in 
the  test.  The  values  measured  were  compared  to  standard  values  and  found  to  be  virtually  identical. 
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Figure  8:  X  and  Y  monitor  chromaticity  calibration  charts 


ANALYSIS 

Below  in  Table  2  is  the  ANOVA  table  for  the  baseline  vehicle  and  the  other  experimental  factors.  The  treated  vehicle  has 
been  excluded  because  of  security  classification.  The  power  of  the  experimental  design  methodology  is  shown  here  in  that 
the  significance  of  individual  factors  and  of  their  interactions  are  available.  Using  this  kind  of  a  test,  one  can  obtain  not  only 
a  model  curve  of  the  detection  probability  versus  any  factor  in  the  test,  but,  one  can  also  obtain  the  relative  importance  of  the 
individual  factors. 


Tests  of  Between-Subjects  Effects 


Dependent  Variable:  RANK  of  RESPONSE 


Source 

Type  IV  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Noncent. 

Parameter 

Observed 

Power8 

Corrected  Model 

185215009b 

89 

2081067.514 

26.782 

.000 

2383.607 

1.000 

Intercept 

1159759015 

1 

1159759015 

14925.408 

.000 

14925.408 

1.000 

SKY_COND 

2918024.068 

2 

1459012.034 

18.777 

.000 

37.553 

1.000 

RANGE 

161301308 

9 

17922367.58 

230.650 

.000 

2075.852 

1.000 

ASPECT 

944347.896 

2 

472173.948 

6.077 

.002 

12.153 

.887 

SKY_COND  *  RANGE 

5751990.053 

18 

319555.003 

4.112 

74.025 

1.000 

SKY_COND*  ASPECT 

2459473.720 

4 

614868.430 

7.913 

.000 

31.652 

.998 

RANGE  *  ASPECT 

6204854.010 

18 

344714.112 

4.436 

.000 

79.853 

1.000 

SKY_COND  *  RANGE 
*  ASPECT 

5397720.861 

36 

149936.691 

1.930 

.001 

69.465 

1.000 

Error 

128521871 

1654 

77703.671 

Total 

1641367780 

1744 

Corrected  Total 

313736880 

1743 

a-  Computed  using  alpha  =  .05 

b.  R  Squared  =  .590  (Adjusted  R  Squared  =  .568) 


Table  2:  ANOVA  of  test  factors 

Figure  9  shows  the  model  generated  logistic  curve  of  the  probability  of  detection  of  the  baseline  LAV.  This  curve  has  the 
effects  of  all  the  various  factors  ‘rolled-up’  into  it. 


LAV  25 


Range  (km) 

Fig.  9  :  Logistic  curve  fit  to  the  model  from  the  subject  responses 


Conclusions 

An  experimental  design  method  was  used  to  design  an  imagery  collection  test  plan  and  a  laboratory  testing 
procedure.  The  probability  of  detection  was  determined  for  the  baseline  and  treated  vehicles.  A  statistical  model  was  made 
of  the  laboratory  results  that  gave  probability  of  detection  versus  range. 

An  advantage  of  using  the  photosimulation  lab  environment  is  that  we  are  able  to  archive  scenes  used  in  the 
simulation.  Therefore,  at  a  later  time  we  are  able  to  rerun  the  same  test  on  a  different  subject  pool.  The  new  subjects  may 
have  a  different  training  and  the  images  may  also  be  modified  by  either  magnification  or  adding  atmospheric  conditions.  This 
provides  tremendous  cost  savings  since  we  do  not  have  to  pay  for  another  field  test. 
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